Draft version February 2, 2008 

Preprint typeset using I^T^jX style cmulatcapj v. 6/22/04 



THE PHOENIX DEEP SURVEY: THE CLUSTERING AND THE ENVIRONMENT OF EXTREMELY RED 

OBJECTS 

A. Georgakakis 1 , J. Afonso 2 , A. M. Hopkins 3 , M. Sullivan 4 , B. Mobasher 5 , L. E. Cram 6 

Draft version February 2, 2008 

ABSTRACT 

In this paper we explore the clustering properties and the environment of the Extremely Red Objects 
(EROs; I — K > 4 mag) detected in a w 180 arcmin 2 deep (Ks w 20 mag) ifs-band survey of a region 
within the Phoenix Deep Survey, an on-going multiwavelength program aiming to investigate the 
nature and the evolution of faint radio sources. Using our complete sample of 289 EROs brighter 
than Ks = 20 mag we estimate a statistically significant (« 3.7a) angular correlation function signal 
with amplitude A w — 8.7t 2 'y x 10~ 3 (assuming w(9) = A w 9~ - 8 , with 9 in deg), consistent with 
earlier work based on smaller samples. This amplitude suggests a clustering length in the range 
r a = 12 — 17/i _1 Mpc, implying that EROs trace regions of enhanced density. Using a novel method 
we further explore the association of EROs with galaxy overdensities by smoothing the iGband galaxy 
distribution using the matched filter algorithm of Postman et al. (1996) and then cross-correlating the 
resulting density maps with the ERO positions. Our analysis provides direct evidence that EROs are 
associated with overdensities at redshifts z > 1. We also exploit the deep radio 1.4 GHz data (limiting 
flux 60/xJy) available to explore the association of EROs and faint radio sources and whether the two 
populations trace similar large scale structures. Cross-correlation of the two samples (after excluding 
17 EROs with radio counterparts) gives a 2a signal only for the sub-sample of high-z radio sources 
(z > 0.6). Although the statistics are poor this suggests that it is the high-z radio sub-sample that 
traces similar structures with EROs. 

Subject headings: Surveys - galaxies: high redshift - galaxies: structure - infrared: galaxies 



1. INTRODUCTION 

The study of the class of Extremely Red Objects 
(EROs; I - K> 4 mag; e.g. Cimatti et al. 2002; Roche 
et al. 2002, 2003) has gained significant impetus over the 
last few years with the realisation that they can provide 
valuable information on galaxy evolution and formation 
scenarios (c.f. Zepf 1997; Barger 1999; Rodighiero et al. 
2001; Daddi et al. 2000; Roche et al. 2002; Vaisanen 
& Johansson 2004a, b). The very red colours of these 
objects suggest that they comprise evolved galaxies at 
redshifts z > 1, the progenitors of present-day ellipti- 
cals. Study of the properties of such high-z systems can 
indeed provide tight constraints on competing elliptical 
galaxy formation scenarios: monolithic collapse early in 
the Universe (zj > 2 — 3) followed by passive evolution 
(e.g. Eggen et al. 1962; Larson 1975) versus hierarchical 
merging and relatively recent formation epochs (Baugh 
et al. 1996; Kauffmann 1996). 

Spectroscopic follow-up observations of either complete 
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ERO samples (Cimatti et al. 2002; Cimatti et al. 2003) 
or individual systems (Dunlop et al. 1996; Spinrad et al. 
1997; Stanford et al. 1997) have indeed confirmed that a 
large fraction of EROs (« 50 per cent) have absorption- 
line spectra. In addition to early type galaxies however, 
these follow-up programs also showed that a significant 
fraction of high-z dust-shrouded active galaxies (star- 
burst or AGNs) are also present among EROs (Cimatti 
et al. 1998; Dey et al. 1999; Afonso et al. 2001; Smith 
et al. 2001; Brusa et al. 2002). 

Although the relative mix between early type and 
dusty systems remains poorly constrained, EROs have 
strong spatial clustering which is comparable if not larger 
than that of present day luminous ellipticals (Daddi et 
al. 2000, 2001, 2003; Firth et al. 2002; Roche et al. 
2002, 2003). This may be interpreted as evidence that 
the populations of distant EROs and nearby ellipticals 
may be evolutionary linked. Study of the clustering of 
EROs, usually quantified via the 2-D correlation func- 
tion, can indeed provide valuable information on the na- 
ture of these systems, their formation and evolution his- 
tory as well as their environment (e.g. Daddi et al. 2001; 
Roche et al. 2002, 2003). Comparison of the clustering 
properties of EROs with those of local E/S0 may pro- 
vide important insights into the evolution of early-type 
galaxies out to z > 1. 

In this paper we explore the clustering properties and 
the environment of EROs detected in a w 180 arcmin 2 
deep (Ks ~ 20 mag) Xs-band survey carried out as part 
of the Phoenix Deep Survey (Hopkins et al. 2003), a 
large multiwavelength program aiming to investigate the 
nature and the evolution of faint (sub-mJy and /iJy) ra- 
dio sources. Compared to previous studies of the clus- 
tering of EROs, the present sample has the advantage of 
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depth combined with wider area providing a large sample 
of EROs to Ks ps 20 mag. This is essential to increase 
the sample size and to improve the statistical reliabil- 
ity of the results compared to previous studies at simi- 
lar depths (e.g. Roche et al. 2002, 2003). We caution 
the reader however, that although our survey is larger 
than previous samples at comparable magnitude limits, 
cosmic variance is an issue and may affect our correla- 
tion amplitude measurements. Much larger, degree scale, 
A-band surveys are required to address this issue (e.g. 
NO AO Deep Wide Survey; Brown et al. 2003). Our con- 
tiguous As-band survey also allows study of the associ- 
ation of EROs with regions of enhanced galaxy density. 
Indeed, although EROs are believed to reside in dense 
regions there is still no direct link between EROs and 
galaxy overdensities. The overlap of our sample with 
the ultra-deep and homogeneous radio observations of 
the Phoenix Deep Survey provides a unique opportunity 
to investigate the association of the radio and the ERO 
populations and how they trace the underlying mass dis- 
tribution. 

Section §|2] presents the data used in this paper and 
describes the selection of the ERO sample. The angular 
correlation function analysis is given in §[21 §01 outlines 
our analysis on the environment of EROs and §|Slpresents 
the cross-correlation of our ERO sample with the faint 
radio population. Finally our results are discussed and 
our main conclusions are summarised in §0 Throughout 
the paper we adopt H Q = TOkms^ 1 Mpc" 1 , SIm = 0.3 
and f^A = 0.7. To allow comparison with previous studies 
all the distant dependent quantities are given in units of 
h = H o /100. 

2. THE PHOENIX DEEP SURVEY 
2.1. Radio and NIR/ 'Optical data 

The Phoenix Deep Survey (PDS 8 ) is an on- going sur- 
vey studying the nature and the evolution of sub-mJy 
and /zJy radio galaxies. Full details of the existing radio, 
optical and near-infrared data can be found in Hopkins et 
al. (2003) and Sullivan et al. (2004); here we summarise 
the salient details. The radio observations were carried 
out at the Australia Telescope Compact Array (ATCA) 
at 1.4 GHz during several campaigns between 1994 and 
2001, covering a 4.56 square degree area centered at 
RA(J2000)=01 h ll m 13 s Dec.(J2000)=-45°45'00". A de- 
tailed description of the radio observations, data reduc- 
tion and source detection are discussed by Hopkins et al. 
(1998, 1999, 2003). The observational strategy adopted 
resulted in a radio map that is homogeneous within the 
central 1 deg radius, with the la rms noise increasing 
from 12/iJy in the most sensitive region to about 90/iJy 
close to the edge of the 4.56 deg 2 field. The radio source 
catalogue consists of a total of 2148 radio sources to a 
limit of 60/xJy (Hopkins et al. 2003). 

As-band near-infrared data of the central region of the 
PDS were obtained using the SofI infrared instrument 
at the 3.6m ESO New Technology Telescope (NTT). 
The observational strategy and details of the data re- 
duction, calibration and source detection are described 
by Sullivan et al. (2004). The As-band mosaic covers a 
13.5 x 13.3arcmin area with a 45min integration time, 
and a central 4.5 x 4.5 arcmin 2 subregion which has an 

8 http://www.atnf.csiro.au/people/ahopkins/phoenix/ 
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Fig. 1. — Ks-band galaxy counts from the SofI survey survey 
of the PDS (filled circles). Also shown are the K's-band number 
counts from Glazebrook et al. (1994; stars) and from the Hawaii 
Deep (HDS; crosses) and Medium Deep (HMDS; open circles) sur- 
veys presented by Gardner, Cowie & Wainscoat (1993). 



effective exposure time of 3h. The completeness limit 
is estimated to be As « 20 mag for the full mosaic and 
As ~ 20.5 mag for the deeper central subregion. 

Deep multicolour imaging (UBVRI) of the PDF has 
been obtained using the Wide Field Imager at the AAT 
on 2001 August 13 and 14 (Wi?/-bands) and the 
Mosaic-II camera on the CTIO-4m telescope on 2002 
September 3 ([/-band), fully overlapping the SofI As- 
band survey. Full details on the data reduction, calibra- 
tion and source detection are again presented in Sullivan 
et al. (2004). In this study we will use the /-band obser- 
vations these being the deepest (/ 24.2 mag; see next 
section) and most appropriate for identifying EROs. 

2.2. The ERO sample 

The A's-band selected catalogue was constructed using 
SEXTRACTOR (Bcrtin & Arnouts 1996). Sources were 
detected on the As-band image and their photometric 
properties were then measured from the seeing matched 
Ks and the /-band frames (see Sullivan et al. 2004 for 
details). To facilitate comparison with previous studies, 
the estimated magnitudes are calibrated to the standard 
Vega based magnitude system. 

The star-galaxy separation was performed using the 

CLASS_STAR flag of SEXTRACTOR (CLASS_STAR> 0.95) 

which is reliable to Ks 16.5. The PDS is selected 
to lie at high Galactic latitude and therefore any con- 
tamination by stars at fainter magnitudes is expected 
to be small. The As-galaxy number counts from these 
observations are plotted in Figure ^ showing that the 
turnover magnitude lies at Ks « 20 mag. Sullivan et al. 
(2004) estimate moderate incompleteness of about 20-30 
per cent in the magnitude range Ks — 19.6 — 20.0 mag. 
In section we argue that this small level of incomplete- 
ness is unlikely to alter our main conclusions. The cen- 
tral deep pointing of our As-band survey has a turnover 
magnitude of Ks « 20.5. In this paper however, we 
consider sources brighter than Ks — 20 mag the approx- 
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imate completeness limit of the entire catalogue. 

EROs are selected to have I — K > 4 mag with the / 
and As-band magnitudes measured in Kron (1980)-like 
elliptical apertures (magauto parameter of Sextractor). 
We note that the 5<r detection threshold for the /-band 
catalogue is 24.2 mag, sufficiently deep to identify EROs 
with Ks = 20 mag. We find a total of 289 EROs to 
this magnitude limit within the 13.5 x 13.3arcmin 2 area 
of our survey. Figure [21 compares the differential counts 
of our EROs with previous studies, suggesting that our 
sample is complete to Ks rs 20 mag. 

A total of 95 radio sources brighter than 60^Jy over- 
lap with the K s-band survey region. Using a matching 
radius of 3 arcsec we find that 17 of them with fluxes in 
the range 65 — 1000/iJy are associated with EROs. The 
properties of these sources will be presented in a future 
paper (Afonso et al. 2004, in preparation) and are not 
considered in the analysis that follows. 

We also attempt to estimate the mean radio properties 
of EROs using the stacking analysis method described 
by Hopkins et al. (2004). In brief we extract sub-images 
from the radio mosaic at the location of the non radio de- 
tected EROs, and construct the weighted average of the 
sub-images (weighted by 1/rms 2 , to maximise the result- 
ing signal-to-noise, since the radio mosaic has a varying 
noise level over the image). Sub-images where low S/N 
emission (> 1.5 a) is present at the location of the non- 
detected source are excluded from the stacking, in order 
to avoid biasing the stacking signal result by the presence 
of a small number of low S/N sources. The stacked image 
has an rms noise of 1.2 /xJy, and a rj 6.6 er detection at 
8.0 /^Jy. While the actual redshift distribution of EROs 
remains poorly constrained, they are believed to lie at 
z > 0.8 (e.g. Cimatti et al. 2002; Cimatti et al. 2003). 
Assuming that the average redshift of these sources is 
z « 1, the inferred average ERO 1.4 GHz luminosity is 
3.6 x 10 22 WHz _1 . For the luminosity estimate we use 
a k-correction assuming a power law spectral energy dis- 
tribution of the form S v cx v~ a with a = 0.8. The lu- 
minosity above corresponds to an average star formation 
rate (assuming the EROs are all star-forming systems) 
of Ri 20 M Q yr _1 , adopting the recent calibration of Bell 
(2003) . Clearly these results rely on a large number of as- 
sumptions, but they serve the useful purpose of providing 
a preliminary estimate for the average radio properties 
of these systems. 

3. TWO POINT CORRELATION FUNCTION OF EROS 

The two-point angular correlation function, w(0), is de- 
fined as the joint probability 5P of finding sources within 
the solid angle elements 8VL\ and 5fl2, separated by an 
angle 9, in the form 



SP = N 2 (1 + w^d^S^ 



(1) 



where N is the mean surface density of galaxies. For a 
random distribution of sources w(9)—0. Therefore, the 
angular correlation function provides a measure of galaxy 
density excess over that expected for a random distribu- 
tion. Various methods for estimating w(9) have been in- 
troduced, as discussed by Infante (1994). In the present 
study, a source is taken as the 'centre' and the number of 
pairs within annular rings is counted. To account for the 
edge effects, Monte Carlo techniques are used by placing 
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Fig. 2. — i\s-band differential galaxy counts for the I — K > 
4 mag EROs in the Soft K s-band survey of the PDS (filled circles). 
Also shown are the K s-band ERO counts from Daddi et al. (2000; 
squares) and Roche et al. (2002; open circles) offset for clarity 
by +0.1 and —0.1 mag respectively. Our sample is complete to 
Ks ta 20 mag. 



random points within the area of the survey. We use the 
w(9) estimator introduced by Landy & Szalay (1993) 



w(9) 



DD - 2DR + RR 
RR ' 



(2) 



where DD and DR are respectively the number of sources 
and random points at separations 9 and 8 + d9 from a 
given galaxy. Similarly, RR is the number of random 
points within the angular interval 9 to 9 + d9 from a 
given random point. 

For a given ERO sample, a total of 100 random cat- 
alogues are generated, each having the same number of 
points as the original data set. The random sets were 
cross-correlated with the galaxy catalogue, giving an av- 
erage value for w(9) at each angular separation. The 
uncertainty in w{9) is determined from the relation 



1 + w{9) 
DD 



(3) 



Finally, before fitting a power law to w(9), we take into 
account a bias arising from the finite boundary of the 
sample. Since the angular correlation function is calcu- 
lated within a region of solid angle f2, the background 
projected density of sources, at a given magnitude limit, 
is effectively N s /Q (where N s is the number of sources 
brighter than the limiting magnitude). However, this 
is an overestimation of the true underlying mean sur- 
face density, because of the positive correlation between 
galaxies in small separations, balanced by negative val- 
ues of w(9) at larger separations. This bias, known as 
the integral constraint, has the effect of reducing the am- 
plitude of the correlation function by 



^ J J w(9)dVL X <m 2 , 



(4) 
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where Q is the solid angle of the survey area. Following 
Roche et al. (2002) we estimate ujq numerically using 
the random-random correlation 



/] RR A w 9 

ERR 



(5) 



where we assume that w(9) is a power-law of the form 
A w 9~ s . The sum in equation [S] is from 1 arsec to 
15arcmin. Adopting 5 — 0.8 we find = 6.9 x A w . 
Having fixed the exponent S to 0.8 the amplitude A w is 
obtained by fitting the function A w 9~ o s — ojq to the ob- 
served w(9) using standard x 2 minimisation procedures 
weighting each point with its error. We estimate the 
w(9) and determine the amplitude A w for different Ks- 
band magnitude limited ERO subsamples. The results 
are summarised in Table ^ The estimated correlation 
function is plotted in Figure 01 For the Ks < 19 and 
< 19.5 mag sub-samples the correlation function signal is 
significant at the 2a level with w(9 < 25") = 0.68 ± 0.30 
and 0.28 ± 0.13 respectively. For the sub-sample with 
Ks < 20 mag the detected signal is significant at the 3.7a 
confidence level (w(9 < 25") = 0.37 ± 0.10). The ampli- 
tudes estimated above must be corrected for contamina- 
tion of the galaxy sample by stars at faint magnitudes 
(Ks > 16.5 mag). The presence of an uncorrelated pop- 
ulation of uniformly distributed stars within the galaxy 
catalogue, reduces A w by the factor 



N ob3 



N, 



obj 



(G) 



where N a bj is the number of objects (both stars and 
galaxies) used to calculate w(9) and N s is the number of 
stars in the catalogue. In the present study, stars were 
identified and removed to K — 16.5 mag (section 2.2). 
At fainter magnitudes, the Milky Way stellar population 
synthesis model described by Robin et al. (2003, 2004) 
was employed to predict the expected number of stars 
within our ERO sample at different if s-band magnitude 
limits. The correction factors estimated from equation 
0are also listed Tabled along with the corrected ampli- 
tudes, A s w . The A s w (with 9 measured in degrees) are con- 
sistent with previous studies at similar magnitude limits 
(e.g. Daddi et al. 2001; Roche et al. 2002, 2003). This is 
demonstrated in Figure 0] plotting the A s w for EROs as a 
function of the limiting ifs-band magnitude. Compared 
to previous ERO surveys to Ks = 20 mag our sample has 
the advantage of larger size and hence more statistically 
reliable results. Cosmic variance however, is likely to be 
an issue for surveys with areal extent similar to our own. 
Daddi et al. (2001) estimated that the relative disper- 
sion on the amplitude of the angular correlation function 
due to this effect is aA m /A w — V 1 A w u>n, where u)q is the 
integral constraint defined in equation 01 For our sur- 
vey geometry using this relation we estimate an uncer- 
tainty to the amplitudes listed in Tabled due to cosmic 
variance of w 20%. We note however, that this result 
is likely to be a lower limit since it does not take into 
account the higher order moments of the galaxy distri- 
bution. For example Barger et al. (1999) and McCracker 
et al. (2000) find a factor of three difference in the sur- 
face density of EROs to K ss 20 mag in their independent 
small area surveys (« 60 arcmin 2 ). Although our sample 



has 3 times larger field of view the result above illustrates 
the effect of cosmic variance to relatively small area sur- 
veys. Much larger, degree scale, if -band samples will be 
able to address this issue (e.g. NOAO Deep Wide Sur- 
vey; Brown et al. 2003). N-body simulations can also 
be used in principle to provide a realistic estimate of the 
cosmic variance effect (e.g. Somerville et al. 2004). This 
calculation however, requires a number of assumptions 
on the redshift distribution and clustering properties of 
EROs and is beyond the scope of this paper. 

We also attempt to quantify the effect of the if s-band 
catalogue incompleteness at faint magnitudes (20-30% in 
the range 19.6-20 mag) to the estimated w(9) of EROs. 
We need a sample that does not suffer from incomplete- 
ness in the above magnitude range to create mock cat- 
alogues by randomly removing 20-30% of the sources at 
faint magnitudes. The angular correlation function for 
each mock catalogue is then estimated to quantify the 
bias introduced by the if s-band catalogue incomplete- 
ness. The above prescription could have been applied 
to the central 4.5 x 4.5 arcmin 2 subregion of our survey 
which is deeper in the ifs-band (Ks as 20.5 mag) and 
thus does not suffer from incompleteness in the 19.6- 
20 mag range. However, only about 30 EROs lie in this 
subregion and therefore the small number statistics do 
not allow us to use this sample. Nevertheless, we apply 
the above method to the Ks < 19.5 mag subsample (to- 
tal of 177 sources, see Table by randomly removing 
20-30% of the sources in the range 19.0-19.5 mag. A to- 
tal of 500 mock catalogues are created and the amplitude 
of the angular correlation function of each one of them 
is estimated. Within the la uncertainties we find no 
difference between the mean A w from the 500 mock re- 
alisations and that quoted in Tabled Extrapolating the 
above result to the Ks < 20 mag we argue that the esti- 
mated amplitude is not going to be significantly affected 
by incompleteness at faint magnitudes. We caution the 
reader however, that the above conclusion holds only if 
the missed objects have the same clustering properties 
as the detected sources. 

The agreement between the angular correlation ampli- 
tudes estimated here and those found in previous stud- 
ies also implies an agreement on the spatial correla- 
tion length r Q assuming a similar redshift distribution, 
which is not unreasonable given the similarity of the sam- 
ple selection. Use of the luminosity function models of 
Daddi et al. (2001) and Roche et al. (2002, 2003) to 
deproject the ERO angular correlation amplitudes and 
to determine their spatial correlation length, r Q , yields 
r a = 12 — 17/i _1 Mpc. The range in r Q depends on 
the adopted luminosity function and clustering evolution 
model. As discussed above the uncertainty in r D due to 
cosmic variance is estimated to be at least sa 20%. We 
also caution the reader that the relatively small extent 
of our survey may result in systematic underestimation 
of the derived correlation length. For example, Daddi et 
al. (2001) estimate this effect to be < 10%. 

4. THE ENVIRONMENT OF EROS 

In this section we study the environment of EROs to 
explore their association with regions of enhanced density 
at relatively high redshifts (z w 1). We quantify this by 
cross-correlating the positions of our ERO sample with 
the smoothed K s-band galaxy density map produced us- 
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TABLE 1 

Clustering amplitudes of the I — K > 4 mag EROs at different 
magnitude limits. The A w is estimated at 1 degree and is not 
corrected for dilution due to stellar contamination of the ERO 
sample at faint magnitudes. The correction factors in the last 

column account for this effect providing an estimate of the 
reduction of the measured A w due to the presence of stars (see 
text for details). The last column gives the clustering amplitudes, 



A"s-band Number of A u 
limit (mag) EROs (xlCr 



19.0 
19.5 
20.0 



100 
177 

289 



7.7 

7.1 

6.8 



+5.6 

3.3 
+2.6 
2.0 
+ 1.6 
1.3 



correction A£ 
factor (xlO 



1.52 
1.34 
1.28 



11.7 



3 ) 
+8.5 



9.5 
8.7 



5.0 
+3.5 
2.7 
+2.1 
1.7 



< 

: jf<19mag 

". . . I 


1 
. 




: " \ 

: K<19.5mag : 
". . . i i ■ , , , ; 


: I-_ T 
: • • . 

i?<20mag ? I 
'. . . 1 . .. 


7>J 



0.001 



0.01 

8 (degrees) 



Fig. 3. — Angular correlation function w(0) for the three mag- 
nitude limited subsamples listed in Table HI The lines are the best 
fit to the observations. 



ing the matched filter algorithm described by Postman 
et al. (1996). 

4.1. Creating the density map 

The matched filter algorithm was developed by Post- 
man et al. (1996) to identify galaxy overdensities us- 
ing photometric data only. It has the advantage that 
it exploits both positional and photometric information 
producing galaxy density maps where spurious galaxy 
fluctuations are suppressed. A drawback of the matched 
filter method is that one must assume a form for the 
cluster luminosity function and its radial profile. Clus- 
ters with the same richness but different intrinsic shape 
or different luminosity function from the adopted ones 
do not have the same likelihood of being detected. 

A detailed description of the matched filter algorithm 
can be found in Postman et al. (1996). In brief, the 
galaxy catalogue is convolved with a filter derived from 
an approximate maximum likelihood estimator obtained 
from a model of the spatial and luminosity distribution 
of galaxies within a cluster. The luminosity weighting 
function (i.e. flux filter) is defined as 



L(m) 



cj){m-m*) 10-° 
b(m) 



(7) 



where <p(m — m*) is the cluster luminosity function, m* 
is the apparent magnitude corresponding to the charac- 
teristic luminosity of the cluster luminosity function and 
b(m) is the surface density of field galaxies with apparent 
magnitude m. We adopt a Schechter form for the cluster 
-ftTs-band luminosity function with parameters a = —1.09 
and M* = —23.53 + blogh estimated by Kochanek et al. 
(2001) for early type galaxies. The term icr - 4 ( m ~ m *) in 
equation [7| is introduced to avoid divergence of the inte- 
gral of L(m) at faint magnitudes in the case of Schechter 
luminosity functions with a < — 1. The spatial weighting 
function (i.e. radial filter) is assumed to follow the form 
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Fig. 4. — The amplitude of the angular correlation function of 
EROs as a function of the limiting ii"s-band magnitude. The filled 
circles are the results from this study, after applying the correction 
for stellar contamination, the squares and the open circles are the 
amplitudes from Daddi et al. (2000) and Roche et al. (2002) 
respectively. 



i 1 1 if r < r 

otherwise, 

where r c is the cluster core radius and r co is an arbitrary 
cutoff radius. Here, we adopt r c = 100 h^ 1 kpc and r co — 
l/i _1 Mpc (e.g. Postman et al. 1996). In practice the 
survey area is binned into pixels of fixed size (for 

the choice of values see below) and the observed galaxy 
distribution is convolved with the filters above producing 
a likelihood map according to the relation 



S(i,j) 



N T 

E 

k=l 



P(r k )L(m k ), 



(8) 



where Nt is the total number of galaxies in the cata- 
logue. The sum above is evaluated for every pixel of 
the density map. Both m* and r c are a function of red- 
shift and hence S(i,j) also depends on redshift through 
these parameters. The redshift dependence of m* also 
includes a fc-correction. Here we adopt a non-evolving el- 
liptical galaxy model obtained from the Bruzual & Char- 
lot (1993) stellar population synthesis code as described 
in Pozzeti, Bruzual & Zamorani (1996). 



6 



Georgakakis et al. 



Fig. 5. — ifs-band density map with parameters tuned to z = 
1.1. The contours delineate the regions with overdensity > 1 (i.e. 
above the mean density of the map). The circles show the positions 
of Ks < 20 mag EROs in our sample. 



We apply the matched filter algorithm to galaxies 
brighter than Ks — 20 mag in our ifs-band survey to 
avoid biases due to incompleteness at fainter magnitudes. 
The galaxy density map S(i,j) is independently esti- 
mated for redshifts between z m i n — 0.5 to z max = 1.2, 
incremented in steps of 0.1. The z max corresponds to the 
redshift where m* becomes comparable to the limiting 
magnitude of the survey. The characteristic luminosity, 
L* , the faint end slope of the luminosity function, a, and 
the cluster core radius, r c , are assumed to remain con- 
stant with redshift. Adopting a passively evolving L* for 
the luminosity function does not alter our main conclu- 
sions but, as discussed below, only shifts z max to higher 
redshifts, z max w 1.6. For the passive evolution we 
adopt the elliptical galaxy model described by Pozzeti, 
Bruzual & Zamorani (1996) which predicts a brighten- 
ing of M* by about 0.8 magnitudes to z ~ 1. This is 
in fair agreement with recent studies on the if-band lu- 
minosity function evolution (Pozzetti et al. 2003; Toft 
et al. 2004; Ellis & Jones 2004). The results presented 
here assume a constant L* with redshift. The pixel size 
of the galaxy density maps at any redshift is taken to 
be ~ 18 arcsec corresponding to a projected cluster core 
radius of r c = 100 hT x kpc at the redshift z = 1. Figure^ 
shows the z = 1.1 density map generated by the method 
above with the positions of EROs overplotted. 

4.2. ERO /density-map cross-correlation 

The cross-correlation function between the Ks < 
20 mag EROs and the ifs-band density map (at a given 
redshift) is estimated using the relation 

N ero s 

WE,Difi)=N— E E^'-^)' (9) 

k i,j 

where N eros is the total number of EROs, Sij is the value 
of the pixel of the density map produced using the 
prescription described in the previous section, S is the 
mean value of the density map and the sum is for all 



pixels with angular separation 9 from a given ERO. The 
uncertainties were estimated from 100 bootstrap resam- 
ples of the EROs. Simulated data sets were generated by 
sampling N points with replacement from the true ERO 
dataset of N points. The cross-correlation function is 
then estimated for each of the bootstrap samples in the 
same fashion as with the real dataset. The standard de- 
viation around the mean for a given angular separation 
9 is the used to estimate the uncertainty in the we,d{@)- 

To assess the expectation in the case of a random dis- 
tribution of points we produce 100 mock ifs-band cat- 
alogues by randomising the positions of the Jfs-band 
galaxies. We then apply the matched filter algorithm to 
produce density maps for each of the independent mock 
catalogues. The (randomised) positions of the EROs in 
the mock catalogues are then cross-correlated with the 
density maps in the same fashion as for the real dataset. 
For each separation 9 the above procedure provides an es- 
timate of the cross-correlation function expected in the 
case of a random distribution of galaxies (i.e. without 
clustering). This procedure also takes into account the 
presence of spurious galaxy overdensities that may be 
produced by the matched filter algorithm. 

Using the procedure above we cross-correlate the Ks < 
20 mag EROs with the density maps produced by the 
matched filter algorithm with parameters tuned at red- 
shifts z = 0.5 - 1.2. The results for the z = 0.7, 0.9 and 
1.1 density maps are plotted in Figure along with the 
random expectation. There is evidence for a statistically 
significant (w 3<r) positive signal for angular separations 
0-60 arcsec in the case of the high redshift density maps 
(e.g. z = 0.9, 1.1). At lower redshifts (e.g. z = 0.7) we 
find no signal above the random expectation. This is fur- 
ther demonstrated in Figure [7| plotting the significance 
above the random expectation of the detected we,d(Q) 
signal for separations 0-60 arcsec (i.e. the first bin in 
Figure EJ) as a function of the redshift that the density 
map was estimated. The significance in this figure is ex- 
pressed in units of standard deviations, er, taking into 
account the uncertainties in both the we,d{0) and the 
random expectation. In Figure \7\ the significance of the 
we,d{9) signal increases with redshift to about 3<r when 
the cross-correlation is performed with density maps gen- 
erated for redshifts z > 1. Adopting a passively evolving 
L* for the luminosity function (see section l4~T)l shifts the 
curve in Figure to higher redshifts with the peak at 
z = 1.2 moving to z = 1.6 but does not qualitatively 
alter the results. This is demonstrated in the insert plot 
of Figure [7| The evidence above suggests that EROs are 
associated with high-z overdensities. 

We also explore if non-ERO (i.e. I — K < 4) if s-band 
selected galaxies with the same magnitude distribution as 
the ERO sample give equally significant cross-correlation 
signal. We produce a total of 100 mock catalogues by 
randomly selecting galaxies from the full ifs-band source 
list with I—K < 4 (i.e. not EROs) and magnitude distri- 
bution similar to that of our Ks < 20 mag ERO sample. 
Each of these mock catalogues are cross-correlated with 
the ifs-band density maps providing, at different red- 
shifts, an estimate of the mean cross-correlation function 
and its rms for each separation. This is then compared 
against the signal for EROs. For the z = 1.2 and 1.1 
maps we find that the ERO signal is higher than that of 
non-ERO galaxies, on average, at the 2.6tx and 2.1a con- 
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Fig. 6. — Cross-correlation function we, d{9) of the Ks < 20 mag 
EROs and the density map produced by the matched filter algo- 
rithm of Postman et al. (1996) at redshifts z = 0.7, 0.9 and 1.1. 
The continuous line is the expectation for a random distribution 
of galaxies and the dashed lines are the lcr uncertainty envelop 
around the mean. 



fidence level respectively. For lower redshifts the EROs 
have cross-correlation signal consistent with that of non- 
EROs within the lcr uncertainties. This is demonstrated 
in Figure \7\ plotting the significance above the random 
expectation of the mean we,d(&) signal for non-ERO 
K s-band selected galaxies for separations 0-60 arcsec as 
a function of the rcdshift that the density map was es- 
timated. Together this all provides direct evidence that 
EROs are associated with regions of enhanced density at 
z> 1. 

5. EROS AND THE FAINT RADIO POPULATION 

Radio galaxies are another class of sources that are 
believed to be good cluster tracers to high-z (e.g. Zir- 
bel 1997; Zanichelli et al. 2001). We exploit the deep 
1.4 GHz radio observations available for our if s-band 
survey to explore the association of EROs with radio 
sources and whether they trace similar structures. These 
are quantified using the two-point cross-correlation func- 
tion w\a,e{9) between radio sources and EROs. 

The cross-correlation function is estimated taking ra- 
dio sources as centres and counting the number of EROs 
around them in successive annuli. This is then compared 
with the expectation for a random distribution by plac- 
ing a total of 40 000 random points in the surveyed area 
and counting the number of pairs between radio sources 
and the random points. The cross-correlation function is 
defined 



wia.e(O) 



£>£>(!. 4, E) N R 
DR(1A, R) N E 



1, 



(10) 



where DD(IA,E), DR(1A, R) is the number of ra- 
dio/ERO and radio/random pairs respectively with sep- 
aration 9 and Nr, N% is the total number of random 
points and EROs respectively. As already discussed, in 
the cross-correlation we have excluded a total of 17 radio 
sources that are associated with EROs. The results for 
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Fig. 7. — Significance above the random expectation of the de- 
tected u>e,d{8) signal for separations 0-60 arcsec as a function of 
the redshift that the density map (used in the cross-correlation) was 
estimated assuming an non-evolving L*. The filled circles are the 
Ks < 20 mag ERO sample (I — K > 4 mag) and the open squares 
are non-ERO (i.e. I — K < 4) K s-band selected galaxies (see text 
for details). The significance is in standard deviations, cr, taking 
into account the uncertainties in both the we d{8) an d the random 
expectation. The insert plot shows the same results (we,d(@) sig- 
nificance versus z) in the case of density maps estimated assuming 
passively evolving L„ luminosity. There is no qualitative difference 
between the non-evolving and passively evolving L„ results except 
from the z max which shifts from 1.2 to 1.6 in latter case. 



the remaining 78 radio sources in our sample are plotted 
in the bottom panel of Figure [S] with the errors being 
Poisson estimates. There is little signal with the esti- 
mated Wia,e(9) consistent with zero. This suggests that 
the bulk of the EROs and the radio source populations 
are probing different structures. 

We further explore this by grouping the radio sources 
into high and low-z sub-samples using the photometric 
redshift estimates presented by Sullivan et al. (2004). 
The low-z sub-sample comprises 35 sources with z p hot < 
0.6 while, the high-z sub-sample includes a total of 43 
sources both without optical counterparts to the limit 
I ps 24 mag (assumed to lie at high redshifts) and with 
Zphot > 0.6. The cross-correlation results using the above 
sub-samples are plotted in the middle (low-z) and top 
panels (high-z) of Figure |S] Although there is no signal 
in the case of the low-z radio sources, the high-z sub- 
sample shows signal for separations 7-30 arcsec, albeit at 
the fa 2cr significance level (iui.4,e(7-30") = 0.34±0.18). 
Although the statistics are poor the evidence above sug- 
gests that the different redshift distributions of the ra- 
dio and ERO populations are responsible for the lack of 
cross-correlation signal in the bottom panel of Figure |H1 
The EROs peak at z > 1 (e.g. Cimatti et al. 2001, 2003; 
Firth et al. 2002; Vaisanen & Johansson 2004a) while 
many of the radio sources to the flux density limit of the 
PDS lie at z < 1 (Georgakakis et al. 1999; Afonso et al. 
2004; Sullivan et al. 2004). 

This is also supported by the cross-correlation of the 
full radio sample with the ifs-band density maps pro- 
duced in section 14.11 The cross-correlation is performed 
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using the method outlined in section 14.21 The expec- 
tation in the case of a random distribution of points is 
quantified by producing 100 mock radio catalogues by 
randomising the positions of the radio sources. Each of 
these random catalogues is cross-correlated with the Ks- 
band density maps providing, for a given separation 9, 
an estimate of the cross-correlation function expected in 
the case of a random distribution of galaxies (i.e. without 
clustering). 

The results are shown in Figure |5] plotting the sig- 
nificance above the random expectation of the detected 
cross-correlation signal for separations 0-60arcsec as a 
function of the redshift that the density map was esti- 
mated (assuming a non-evolving L*). The significance in 
this figure is expressed in units of standard deviations, a. 
In FigureElthe significance of the cross-correlation signal 
decreases with redshift falling below the « 3c when the 
cross-correlation is performed with density maps gener- 
ated for redshifts z > 0.7. Adopting a passively evolving 
L* for the luminosity function (see section l4~TJl does not 
qualitatively change our results but only shifts to higher- 
z the point that the significance of the cross-correlation 
signal falls below w 3o\ This is demonstrated in the 
insert plot of Figure 

The trend for radio sources in Figure is opposite to 
that for EROs in Figuredsuggesting that the bulk of the 
radio and the ERO populations are tracing structures at 
different redshifts. 

6. DISCUSSION AND SUMMARY 

In this paper we explore the clustering properties of 
Ks < 20 mag EROs (I — K > 4 mag) using a i\Vband 
sample covering w 180 arcmin 2 and overlapping with the 
PDS region. We use angular correlation function analy- 
sis to estimate a statistically significant clustering signal 
(> 3cr) with amplitude consistent, within the errors, with 
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Fig. 9. — Significance above the random expectation of the cross- 
correlation function, 101.4,0(1?), between the faint radio sources and 
the if s-band density maps at different redshifts produced as de- 
scribed in section [I] The density maps are estimated assuming a 
non-evolving L* for the luminosity function. We plot the w\,4jj{d) 
signal for separations 0— 60arcsec expressed in standard deviations, 
a, taking into account the uncertainties in both the wi.4 d(0) and 
the random expectation. The insert plot shows the same results 
(wia,d(8) significance versus 2) in the case of density maps esti- 
mated assuming passively evolving L* luminosity. There is no qual- 
itative difference between the non-evolving and passively evolving 
L* results. 



previous studies (Roche et al. 2002, 2003; Daddi et al. 
2000). The advantage of our study is that our sample 
size is larger than previous surveys to Ks — 20 mag (e.g. 
Roche et al. 2002, 2003) providing better statistical reli- 
ability. Nevertheless, cosmic variance remains an issue in 
our survey. The w(0) amplitudes estimated here are con- 
sistent with 3-D clustering lengths r a = 12 — 17 h -1 Mpc 
depending on the adopted model luminosity function and 
the evolution of the clustering (Daddi et al. 2001; Roche 
et al. 2002, 2003). These r Q values are in-between 
those of present-day ellipticals (e.g. Guzzo et al. 1997) 
and Abell clusters (Abadi, Lambas & Muriel 1998) and 
comparable to those estimated for hard X-ray selected 
sources (Basilakos et al. 2004) and SCUBA sources (Al- 
maini et al. 2003). Daddi et al. (2002) also discuss 
evidence that EROs with optical/NIR colours suggest- 
ing old stellar population are significantly more clustered 
than those showing evidence for dusty starbursts. 

A number of studies also suggest a dependence of the 
correlation length on the galaxy absolute luminosity (e.g. 
Willmer, da Costa & Pellegrini 1998; Norberg et al. 2002; 
Zehavi 2002; Brown et al. 2003). For example, Brown et 
al. (2003) estimated the correlation length of z < 1 red 
(B w -R> 1.44) galaxies in the NOAO Deep Wide Field 
Survey and found that systems with L > L* have higher 
clustering lengths (r ~ 10 /i -1 Mpc) than < L* galaxies 
(r a « 5/i Mpc). Similar results are obtained for the 
early type galaxies in the 2dF Galaxy Redshit Survey 
(Norberg et al. 2002). The EROs identified in our sam- 
ple comprise systems with L < L* . For example at z = 1 
and 1.5 the magnitude limit Ks = 20 mag corresponds 
to about 0.2 L* and 0.5 L* respectively, assuming a pas- 
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sively evolving elliptical galaxy model. The large cluster- 
ing lengths of Ks w 20 mag EROs (r a = 12-17/i," 1 Mpc) 
are therefore difficult to reconcile with the above r a esti- 
mates for L < L* early type galaxies at z < 1. This may 
be due to uncertainties in the model redshift distribu- 
tion of EROs used to deproject the angular correlation 
function amplitudes. Combination of photometric and 
spectroscopic redshifts for complete ERO samples are es- 
sential to refine these models and to better understand 
the clustering properties of this population. 

The evidence above on the large correlation length 
of EROs may suggest these sources are tracing high 
density regions. We further explore the association of 
EROs with galaxy overdensitics using a novel method: 
we first smooth the ifs-band galaxy distribution using 
the matched filter algorithm of Postman et al. (1996) 
and then cross-correlate the resulting density maps with 
the ERO positions. The matched filter algorithm pa- 
rameters are tuned to produce density maps each one of 
which is sensitive to galaxy clusters at a different redshift 
in the range 0.5-1.2. We find a statistically significant 
cross-correlation signal (> 3<r compared to the random 
distribution) only for density maps tuned to z > 1 clus- 
ters. At these redshifts, EROs also have higher cross- 
correlation signal, albeit at the 2 — 2.5a confidence level, 
than the non-ERO ifs-band selected galaxies with the 
same magnitude distribution. This provides direct evi- 
dence that EROs are, on average, associated with regions 
of enhanced galaxy density at redshifts z > 1. Previ- 
ous studies also claim that EROs lie within rich clus- 
ter regions at high-z. Daddi et al. (2000) and Roche 
et al. (2002) have identified ERO overdensities within 
their surveys and argued that these may be associated 
with massive clusters at z > 1. Vaisanen & Johansson 
(2004b) also report overdesnities of EROs in the vicinity 
of faint mid-IR ISO sources and argue that these may be 
associated with high-z clusters. Similar enhanced ERO 
number densities have been reported in high-z AGN and 
QSO fields (e.g. McCarthy, Presson & West 1992; Hu & 
Ridgeway 1994; Chapman, McCarthy & Persson 2000; 
Hall et al. 2001; Best et al. 2003; Wold et al. 2003), al- 
though is not yet clear if these ERO overdensities are in- 
deed associated with clusters linked to the central AGN. 
Our analysis provides direct evidence that EROs are as- 
sociated with high-z overdense regions and confirms the 
claims of the studies above. 

Also, our finding that EROs trace dense regions at 
z > 1 is consistent with previous studies on the red- 
shift distribution of EROs. The red I — K colours of 
these systems can only be explained by either early type 
galaxies or dusty starbursts at z > 0.8 (e.g. Pozzetti & 
Mannucci 2000; Vaisanene & Johansson 2004a; Roche 
et al. 2002, 2003). Luminosity function models used 
to interpret the observed ERO counts and their cluster- 
ing properties produce spiky redshift distributions with a 
peak at z w 1 and a long tail extending to high-z (Daddi 
et al. 2001; Roche et al. 2003). Spectroscopic and pho- 
tometric redshifts for EROs also show that most of them 
are associated with z > 1 ellipticals or dusty starbursts, 
although the relative mix of the two populations remain 
poorly constrained (e.g. Cimatti et al. 1998; Dey 1999; 
Piere et al. 2001; Smith et al. 2001; Afonso et al. 2001; 
Cimatti et al. 2001; Daddi et al. 2003; Martini 2001). 

We also investigate the association between EROs and 



faint radio sources and how they trace large scale struc- 
tures by exploiting the deep radio data available for 
ifs-band survey. We find that only a small fraction of 
Ks < 20 mag EROs have radio counterparts to the 60^Jy 
limit of the radio data (17 out of 289). This is much lower 
than the identification fraction reported by Smail et al. 
(2002; 21/68) and Roche et al. (2003; 7/31) which is at- 
tributed to the deeper radio observations in those studies. 
Indeed, Smail et al. (2002) find that the number of EROs 
detected at radio wavelengths rapidly increases at faint 
flux densities, doubling below 40^iJy. Using stacking 
analysis we estimate a mean radio flux density for EROs 
of w 8 ^Jy, about 1 dex lower than the limit of our sur- 
vey. Much deeper observations are therefore required to 
identify the bulk of the radio counterparts for EROs. 

Cross-correlating the radio and ERO positions (after 
excluding the 17 EROs with 1.4 GHz emission) does not 
produce any signal, suggesting little association between 
the two populations. However, for the sub-sample of ra- 
dio sources with photometric redshifts z > 0.6 or no op- 
tical counterparts (assumed to lie at high-z) we get a 
marginally significant signal at the w 2a level. Although 
the small number statistics (only 43 radio sources fulfill- 
ing the above criteria) hamper a secure interpretation, 
the evidence above suggests that it is the high-z radio 
population that appears to trace similar large scale struc- 
tures with EROs. 

A large fraction of the radio population however, to 
the limit of our survey, is associated with moderate 
and low-z starburst systems (Georgakakis et al. 1999; 
Afonso et al. 2004) tracing structures at z < 1. Cross- 
correlation of the full radio sample with the ifs-band 
density maps gives a statistical significant signal (> 3a) 
only at z < 0.7, contrary to the ERO population with 
a cross-correlation signal that peaks at z > 1. This ex- 
plains why cross-correlation of the full radio and ERO 
samples does not produces a statistically significant sig- 
nal. 

Summarising our conclusions: 

• using angular correlation function analysis we es- 
timate a statistically significant signal (> 3a) for 
Ks < 20 mag EROs. The derived correlation func- 
tion amplitude is consistent with previous stud- 
ies that used smaller sample sizes. This ampli- 
tude translates to clustering lengths in the range 
r Q = 12- 17 h~ l Mpc. 

• cross-correlation of the ERO positions with the 
Ks-b&nd density maps gives a statistically signif- 
icant signal only for the z > 1 maps. This cross- 
correlation signal is higher, albeit at the 2 — 2.5a 
level, from that obtained for non-ERO galaxies 
with the same magnitude distribution as our ERO 
sample. We argue that this is direct evidence that 
EROs are associated with regions of enhanced den- 
sity at redshifts z > 1. 

• 17 of the 289 EROs with K < 20 mag show ra- 
dio emission with flux densities in the range 65 — 
lOOO^Jy. Using stacking analysis we estimate a 
mean radio flux density of « 8 ^Jy for EROs. 

• cross-correlation of the radio and the ERO samples 
(after excluding the 17 EROs with radio counter- 
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parts) gives a 2er signal only for the sub-sample 
of high- z radio sources (z > 0.6). Although the 
statistics are poor this suggests that the bulk of 
the radio and the ERO populations trace different 
structures. Indeed, cross-correlation of the radio 
positions with the Jfs-band density maps gives a 
statistically significant signal only for the z < 0.7 
maps, contrary to EROs. 
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